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ABSTRACT 



The National Assessment of Educational Progress (NAEP) is 
the only nationally representative and continuing assessment of what students 
in the United States know and can do in various academic subjects. The 1996 
NAEP in mathematics assessed the current level of mathematical achievement as 
a mechanism for informing education reform. In 1996, 44 states, the District 
of Columbia, Guam, and the Department of Defense schools took part in the 
NAEP state mathematics assessment program. The NAEP 1996 state mathematics 
assessment was at grade 4 and grade 8, although grades 4, 8, and 12 were 
assessed at the national level. The 1996 state mathematics assessment covered 
the five content strands: (1) Number Sense, Properties, and Operations; (2) 

Measurement; (3) Geometry and Spatial Sense; (4) Data Analysis, Statistics, 
and Probability; and (5) Algebra and Functions. In New Hampshire, 212 
students in 12 nonpublic schools were assessed at the eighth-grade level. (To 
ensure comparability across jurisdictions, the National Center for Education 
Statistics, has established guidelines for school and student participation 
rates. In 1996, New Hampshire participated at grade 8 only, with both public 
and nonpublic school samples. The grade 8 public school sample did not meet 
the guidelines for publication, due to low participation rate; however, the 
nonpublic school sample was sufficient to meet the guidelines for 
publication.) This report describes the mathematics proficiency of New 
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Hampshire eighth-grade nonpublic school students, compares their overall 
performance to students in the Northeast region of the United States and the 
entire United States (using data from the NAEP national assessment) , presents 
the average proficiency for the five content strands, and summarizes the 
performance of subpopulations (gender, race/ethnicity, parents 1 educational 
level, Title I participation, and free/reduced lunch program eligibility) . To 
provide a context for the assessment data, participating students, their 
mathematics teachers, and principals completed questionnaires which focused 
on: school characteristics (attendance) ; instructional content (curriculum 
coverage, standards; amount of homework) ; delivery of mathematics instruction 
and its characteristics; use of technology in mathematics instruction; 
students' own views about mathematics; and conditions facilitating 
mathematics learning (hours of television watched, parental support, home 
influences) . (ASK) 
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I ew Hampshire was one of the original participants in the state-level National 
Assessment of Educational Progress (NAEP) in 1990. Results were reported for New Hampshire 
in 1990 (for mathematics at grade 8), in 1992 (for mathematics at grades 4 and 8, and reading at 
grade 4), and again in 1994 (for reading at grade 4). In 1994, New Hampshire participated with 
both public and nonpublic school samples, but met the participation rate requirements for 
publication of results only for their public school sample. 

To ensure comparability across jurisdictions, NCES has established guidelines for school 
and student participation rates. Appendix A highlights these guidelines, which are applied 
separately for public and nonpublic schools. For jurisdictions failing to meet the initial school 
participation rate of 70 percent for either public or nonpublic schools, appropriate results are not 
reported. Jurisdictions that exceed the 70 percent rate but fail to meet others of these guidelines 
are noted in tables and figures in NAEP reports containing state-by-state results. 

In 1996, New Hampshire again participated at grade 8 only, but with both public and 
nonpublic school samples. The grade 8 public school sample did not meet the guidelines for 
publication, due to low participation rate (see Appendix A); however, the nonpublic school 
sample was sufficient to meet the guidelines for publication. 

The results of the NAEP 1996 mathematics assessment in New Hampshire’s nonpublic 
schools are presented here. The total sample size for nonpublic schools is modest, and there are 
a small number of results to report. Only those results based on pre-established NAEP minimum 
sample sizes are reported. The results that can be reported are students’ average scale scores and 
percentages of students reaching the Basic , Proficient , and Advanced achievement levels. Also 
included are tables showing the demographic composition of the sample, and the participation 
rates for sample components. 

A full set of Appendices is included: 

Appendix A Reporting the NAEP 1996 Mathematics Results 

Appendix B NAEP 1996 Mathematics Assessment 

Appendix C Technical Appendix 

Appendix D Setting the Achievement Levels 
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OVERVIEW 



^^onitoring the performance of students in subjects such as mathematics is a key 
concern of the citizens, policy makers, and educators involved with educational reform efforts. 
The 1996 National Assessment of Educational Progress (NAEP) in mathematics (as well as the 
two previous NAEP assessments in mathematics in 1990 and 1992) assessed the current level of 
mathematics achievement as a mechanism for informing education reform. In 1996, New 
Hampshire participated in NAEP at grade 8 but only the nonpublic schools met the participation 
guidelines. This report contains those results. 

What Is NAEP? 

The National Assessment of Educational Progress (NAEP) is the only nationally 
representative and continuing assessment of what students in the United States know and can do 
in various academic subjects. NAEP is authorized by Congress and directed by the National 
Center of Education Statistics of the U.S. Department of Education. The National Assessment 
Governing Board (NAGB), an independent body, provides policy guidance for NAEP. 

Since its inception in 1969, NAEP’s mission has been to collect, analyze, and produce 
valid and reliable information about the academic performance of students in the United States in 
various learning areas. In 1990, the mission of NAEP was expanded to provide state-by-state 
results on academic achievement. Participation in the state-by- state NAEP is voluntary and has 
grown from 40 states and territories in 1990 to 48 in 1996. 

NAEP has also become a valuable tool in tracking progress towards the National 
Education Goals. The subjects assessed by NAEP are those highlighted at the 1989 Education 
Summit and later legislation. 1 The NAEP 1996 assessment in mathematics marks the third time 
the subject has been assessed with the new framework in the 1990s, enabling policy makers and 
educators to track mathematics achievement since the release of the National Council of 
Teachers of Mathematics (NCTM) Curriculum and Evaluation Standards for School 
Mathematics 2 in 1989. 



1 Executive Office of the President. National Goals for Education. (Washington, DC: Government Printing Office, 1990); Goals 
2000: Educate America Act, Pub. L. No. 103-227 (1994). 

2 National Council of Teachers of Mathematics. Curriculum and Evaluation Standards for School Mathematics. (Reston, VA: 
NCTM, 1989). 
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What Was Assessed? 

The NAEP assessment measures a mathematics domain containing five mathematics 
strands (number sense, properties, and operations; measurement; geometry and spatial sense; 
data analysis, statistics, and probability; and algebra and functions). Questions involving content 
from one or more of the strands are also categorized according to the domains of mathematical 
abilities and mathematical power. The first of these, mathematical abilities, describes the nature 
of the knowledge or processes involved in successfully handling the task presented by the 
question. It may reflect conceptual understanding, procedural knowledge, or a combination of 
both in problem solving. The second domain, mathematical power, reflects processes stressed as 
major goals of the mathematical curriculum. Mathematical power refers to the students’ ability 
to reason, to communicate, and to make connections of concepts and skills across mathematical 
strands, or from mathematics to other curricular areas. 

The mathematics framework for the NAEP 1996 assessment is a revision of that used in 
the 1990 and 1992 assessments. Changes were made to the earlier framework in light of the 
NCTM Standards and changes taking place in school mathematics programs. The previous 
NAEP mathematics framework was refined and sharpened so that the 1996 assessment would: 

(1) more adequately reflect recent curricular emphases and objects and yet (2) maintain a 
connection with the 1990 and 1992 assessments to measure trends in student performance. Prior 
to the 1996 assessment, investigations were conducted to ensure that results from the assessment 
could be reported on the existing NAEP mathematics scale. The conclusion drawn from these 
investigations was that results from the 1990, 1992, and 1996 assessments could be reported on a 
common scale and trends in mathematics performance since 1990 examined. Appendix B briefly 
highlights selected changes in the current NAEP mathematics framework. 

The conception of mathematical power as reasoning, connections, and communication 
has played an increasingly important role in measuring student achievement. In 1990, the NAEP 
assessment included short constructed-response questions as a way to begin addressing 
mathematical communication. In 1992, the extended constructed-response questions included on 
the assessment required students not only to communicate their ideas but also to demonstrate the 
reasoning they used to solve problems. The 1996 assessment continued to emphasize 
mathematical power by including constructed-response questions focusing on reasoning and 
communication and by requiring students to connect their learning across mathematical content 
strands. These connections were addressed within individual questions reaching across content 
strands and by families of questions contained within a single content strand. 

In real life, few mathematical situations can be clearly classified as belonging to one 
content strand or another, and few situations require only one fact of mathematics thinking. 
Therefore, many of the questions are classified in a number of ways. In addition to being 
classified by all applicable content strands, each question was classified by its assessment of 
applicable mathematical abilities (procedural knowledge, conceptual understanding, and problem 
solving) and mathematical powers (reasoning, communication, and connections). The content 
strands, mathematical abilities, and mathematical power combine to form the framework for the 
NAEP assessment. (A brief description of the five content strands is presented in Appendix B.) 

9 
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The framework continued the shift from multiple-choice questions to questions that 
required students to construct responses, in 1996, more than 50 percent of student assessment 
time was devoted to constructed-response questions. Two types of constructed-response 
questions were included — (1) short constructed-response questions that required students to 
provide answers to computation problems or to describe solutions in one or two sentences, and 
(2) extended constructed-response questions that required students to provide longer responses 
when answering the questions. 

Who Was Assessed? 

Eighth-Grade School and Student Characteristics 

Table 1 provides a profile of the demographic characteristics of the eighth-grade students 
in nonpublic schools in New Hampshire, the Northeast region, and the nation. This report 
contains assessment results for nonpublic school students only. New Hampshire participated in 
the NAEP mathematics assessment in 1990 and 1992 and met the minimum guidelines for 
publication for their public school results in both years. For the 1996 NAEP, although both 
public and nonpublic schools participated, only nonpublic schools met the guidelines for 
publication of results. As described in Appendix A, the state data and the regional and national 
data are drawn from separate samples. 




THE NAEP 1996 STATE ASSESSMENT IN MATHEMATICS 



10 



5 



New Hampshire 



THE NATION'S 
REPORT 



CARO 



ivaep 



TABLE 1 — GRADE 8 



IQQfi 

State Assessment 


Profile of Students in New Hampshire, the Northeast Region, and 
the Nation 


Demographic Subgroups 


Nonpublfc Schools 


Percentage 



RACE/ETHNICITY 






New Hampshire 


White 


■ 93 ( 2.6) 




Black 


2 ( 0.8) 




Hispanic 


4(1.7) 




Asian/Pacific Islander 


1 ("") 
0 (****) 
79 ( 8.4) 




American Indian 


Northeast 


White 




Black 


1 1 (****) 




Hispanic 


8(1.8) 




Asian/Pacific Islander 


1 (0.5) 




American Indian 


i m 

80 ( 3.4) 


Nation 


White 




Black 


7(2.5) 




Hispanic 


9 ( 2.0) 




Asian/Pacific Islander 


3 ( 0.7) 




American Indian 


1 ( 0.3) 


TYPE OF LOCATION* 




New Hampshire 


Central city 


45(16.2) 




Urban fringe/Large town 


9 (****) 




Rural/Small town 


46(13.8) 


Nation 


Central city 


69 (6.8) 




Urban fringe/Large town 


22(6.1) 




Rural/Small town 


9(4.2) 


PARENTS' EDUCATION 




New Hampshire 


Did not finish high school 


i m 




Graduated from high school 


10(4.9) 




Some education after high school 


12(3.3) 




Graduated from college 


74 ( 6.5) 




1 don’t know 


4(1.2) 


Northeast 


Did not finish high school 


2 ( 0.7) 




Graduated from high school 


15(3.8) 




Some education after high school 


16 ( 1 . 8 ) 




Graduated from college 


56(8.1) 




1 don’t know 


11(4.4) 


Nation 


Did not finish high school 


2(0.4) 




Graduated from high school 


13 ( 1.5) 




Some education after high school 


16(0.9) 




Graduate from college 


61 ( 2.7) 




1 don’t know 


9(1.6) 


GENDER 






New Hampshire 


Male 


53 ( 9.5) 




Female 


47 ( 9.5) 


Northeast 


Male 


56 ( 6.0) 




Female 


44 ( 6.0) 


Nation 


Male 


53 ( 2.1) 




Female 


47(2.1) 



a 
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TABLE 1 — GRADE 8 (continued) 


REPORT 

CARD 


waep 




1996 




r- 


Profile of Students in New Hampshire, the Northeast Region, and 


State Assessment 


the Nation 



Demographic Subgroups 



Nonpublic Schools 



Percentage 



TITLE 1 

New Hampshire 


Participated 
Did not participate 


2 ( — ) 
98 ( — ) 


Northeast 


Participated 
Did not participate 


2 (****) 
98 ( — ) 


Nation 


Participated 


2(1.2) 




Did not participate 


98 ( 1 .2) 


FREE/REDUCED-PRICE LUNCH 


1 (****) 

49 (17.5) 

50 (17.4) 


New Hampshire 


Eligible 
Not eligible 

Information not available 


Northeast 


Eligible 
Not eligible 

Information not available 


4 (****) 

47 (16.7) 

48 (16.8) 


Nation 


Eligible 
Not eligible 

Information not available 


4(1.5) 
53 ( 7.5) 
43 ( 7.6) 



of interest, the value for the entire population is within ± 2 standard errors of the estimate for the sample. In comparing two 
estimates, one must use the standard error of the difference (see Appendix A for details). The percentages for Race/Ethnicity may not 
add to 100 percent because some students categorized themselves as “Other.” * Characteristics of the school sample do not permit 
reliable regional results for type of location. **** Standard error estimates cannot be accurately determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Mathematics 
Assessment. 
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Schools and Students Assessed 

For public schools in 1996, the weighted school participation rate before substitution was 
69 percent for New Hampshire; a level of 70 percent was required for publication of public 
school results. Table 2 summarizes participation data for nonpublic schools and students 
sampled in New Hampshire for the 1996 state assessment program in mathematics. 3 Also 
included is the participation profile for public schools in 1996. 

In New Hampshire, 12 nonpublic schools participated in the 1996 eighth-grade 
mathematics assessment. The weighted school participation rate after substitution in 1996 was 
85 percent for nonpublic schools, which means that the eighth-grade students in this sample were 
directly representative of 85 percent of all the eighth-grade public school students in New 
Hampshire. 

In New Hampshire 212 nonpublic school eighth-grade students were assessed in 1996. 
The weighted student participation rate was 96 percent for nonpublic schools. This means that 
the sample of eighth-grade students who took part in the assessment was directly representative 
of 96 percent of the eligible nonpublic school student population in participating schools in New 
Hampshire (that is, all students from the population represented by the participating schools, 
minus those students excluded from the assessment). The overall weighted response rate (school 
rate times student rate) was 82 percent for nonpublic schools. This means that the sample of 
students who participated in the assessment was directly representative of 82 percent of the 
eligible eighth-grade nonpublic school population in New Hampshire. 

In each school, a random sample of students was selected to participate in the 
assessment. In 1996, on the basis of sample estimates, 1 percent of the eighth-grade nonpublic 
school population in New Hampshire was classified as having limited English proficiency (LEP). 
In nonpublic schools at the eighth grade, 3 percent of the students had an Individualized 
Education Plan (IEP). An IEP is a plan written for a student who has been determined to be 
eligible for special education. The IEP typically sets forth goals and objectives for the student 
and describes a program of activities and/or related services necessary to achieve the goals and 
objectives. 



3 For a detailed discussion of the NCES guidelines for sample participation, see Appendix A of this report or the Technical Report of 
the NAEP 1996 State Assessment Program in Mathematics. (Washington, DC: National Center for Education Statistics, 1997). 
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TABLE 2 — GRADE 8 



State Assessment 



£ Profile of the Population Assessed in New Hampshire 



1996 



Nonpublic Schools 



SCHOOL PARTICIPATION 

Weighted school participation rate before substitution 


85% 


Weighted school participation rate after substitution 


85% 


Number of schools originally sampled 


19 


Number of schools not eligible 


4 


Number of schools in original sample participating 


12 


Number of substitute schools provided 


2 


Number of substitute schools participating 


0 


Total number of participating schools 


12 


STUDENT PARTICIPA TtON 




Weighted student participation rate after makeups 


96% 


Number of students selected to participate in the assessment 


224 


Number of students withdrawn from the assessment 


4 


Percentage of students who were of Limited English Proficiency 


1% 


Percentage of students excluded from the assessment due to 
Limited English Proficiency 


1% 


Percentage of students who had an Individualized Education Plan 


3% 


Percentage of students excluded from the assessment due to 
Individualized Education Plan status 


0% 


Number of students to be assessed 


219 


Number of students assessed 


212 


Overall weighted response rate 


82% 
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RESULTS 

FOR 

NONPUBLIC 

SCHOOLS 



NAEP Mathematics Results for New Hampshire 

T he NAEP 1996 state assessment program in mathematics provides a wealth of 
information on the mathematical abilities and skills of the fourth-and eighth-grade students in 
participating jurisdictions. To maximize usefulness to policy makers, educators, parents, and 
other interested parties, the NAEP results are presented both as average scale scores on the 
NAEP mathematics scale (Table 3) and in terms of the percentage of students attaining NAEP 
mathematics achievement levels (Table 4). Thus, NAEP results not only provide information 
about what students know and can do, but also indicate whether their achievement meets 
expectations of what students should know and should be able to do. Furthermore, the 
descriptions of skills and abilities expected of students at each achievement level help make the 
reporting of assessment results more meaningful. 

Interpreting NAEP Results 

This report describes mathematics performance for eighth graders in New Hampshire 
nonpublic schools and compares the results with those of eighth grade students in nonpublic 
schools in the Northeast region and in the nation. 

Because the percentages of students and their average mathematics scale scores are 
based on samples — rather than on the entire population of eighth graders in a jurisdiction — the 
numbers reported are necessarily estimates. As such, they are subject to a measure of 
uncertainty, reflected in the standard error of the estimate. When the percentages or average 
scale scores of certain groups are compared, it is essential to take the standard error into account, 
rather than to rely solely on observed similarities or differences. Therefore, the comparisons 
discussed in this report are based on statistical tests that consider both the magnitude of the 
difference between the means or percentages and the standard errors of those statistics. 

The statistical tests determine whether the evidence — based on the data from the groups 
in the sample — is strong enough to conclude that the averages or percentages are really different 
for those groups in the population. If the evidence is strong (i.e., the difference is statistically 
significant), the report describes the group averages or percentages as being different (e.g., the 
states’ students performed higher than or lower than the nation’s students) regardless of 
whether the sample averages or sample percentages appear to be about the same or not. If the 
evidence is not sufficiently strong (i.e., the difference is not statistically significant), the averages 
or percentages are described as being not significantly different — again, regardless of whether 
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the sample averages or sample percentages appear to be about the same or widely discrepant. 
The reader is cautioned to rely on the results of the statistical tests rather than on the apparent 
magnitude of the difference between sample averages or percentages to determine whether those 
sample differences are likely to represent actual differences between the groups in the 
population. The statistical tests are discussed in greater detail in Appendix A. 

The Mathematics Scale 

Students’ responses to the NAEP 1996 mathematics assessment were analyzed to 
determine the percentage of students responding correctly to each multiple-choice question and 
the percentage of students responding in each of several score categories for constructed- 
response questions. Item response theory (IRT) methods were used to produce across-grade 
scales that summarized results for each of the five mathematics content strands discussed earlier. 
Each of the content-strand scales, which range from 0 to 500, was linked to its corresponding 
scale from 1990 and 1992 through IRT equating. 

An overall composite scale was developed by weighting the separate content-strand 
scales based on the relative importance to each content strand in the NAEP mathematics 
framework. The resulting scale, which was also linked to the 1990 and 1992 mathematics 
composite scales, is the reporting metric used to present results. (Details of the scaling 
procedures are presented in the NAEP 1996 Technical Report and in the Technical Report of the 
NAEP 1996 State Assessment Program in Mathematics.) 
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TABLE 3 — GRADE 8 


— Vj 

essment 


Distribution of Mathematics Scale Scores for Students in 
Nonpublic Schools 




Average 
Scale Score 


10th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


90th 

Percentile 


Nonpublic Schools 
1996 New Hampshire 
Northeast 
Nation 


293 ( 4.3) 
281 ( 5.1)! 
284 ( 2.4) 


257 ( 5.2) 274 ( 4.0) 293 ( 3.5) 313 ( 5.9) 329 ( 4.7) 

243(12.4)! 260(6.3)! 283(6.4)! 302(5.9)! 320(8.0)! 

242(4.4) 262(4.6) 286(1.8) 307( 2.4) 326( 3.1) 



The NAEP mathematics scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It can be said with 
about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 standard errors of 
the estimate for the sample. In comparing two estimates, one must use the standard error of the difference (see Appendix A for 
details). ! Interpret with caution — the nature of the sample does not allow accurate determination of the variability of this statistic. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Mathematics 
Assessment. 

• In New Hampshire, the average mathematics scale score of students attending 
nonpublic schools (293) was not significantly different from* that of nonpublic 
school students across the nation (284). 



Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 
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Mathematics Achievement Levels 

Results for the NAEP 1996 assessment in mathematics are also reported using the 
mathematics achievement levels that were authorized by the NAEP legislation and adopted by 
the National Assessment Governing Board. The achievement levels are based on collective 
judgments about what students should know and be able to do relative to the body of content 
reflected in the NAEP mathematics assessment. Three levels were defined for each grade — 
Basic , Proficient , and Advanced . The levels were defined by a broadly representative panel of 
teachers, education specialists, and members of the general public. 

For reporting purposes, the achievement levels for each grade are placed on the NAEP 
mathematics scale. Figure 1 presents the policy definitions of the achievement levels, while 
Figure 2 contains specific descriptions for the levels at grade 8. 



Figure 1. Policy Definitions of NAEP Achievement Levels 



Basic 


This level denotes partial mastery of 
prerequisite knowledge and skills that 
are fundamental for proficient work 
at each grade. 


Proficient 


This level represents solid academic performance 
for each grade assessed. Students reaching this 
level have demonstrated competency over 
challenging subject matter, including subject- 
matter knowledge, application of such knowledge 
to real-world situations, and analytical skills 
appropriate to the subject matter. 


Advanced 


This level signifies superior performance. 



It should be noted that setting achievement levels is a relatively new process for NAEP, 
and it is still in transition. Some evaluations have concluded that the percentage of students at 
certain levels may be underestimated. 4 On the other hand, critiques of those evaluations have 
asserted that the weight of the empirical evidence does not support such conclusions. 5 A further 
review is currently being conducted by the National Academy of Sciences. 



4 General Accounting Office. Educational Achievement Standards : NAGB 's Approach Yields Misleading Interpretations. 
(Washington, DC: General Accounting Office, 1993): National Academy of Education. Setting Performance Standards for 
Student Achievement. A report of the National Academy of Education Panel on the evaluation of the NAEP Trial State 
Assessment: An evaluation of the 1 992 achievement levels. (Stanford, CA: National Academy of Education, 1993). 

5 Cizek, G. Reactions to the National Academy of Education report. (Washington, DC: National Assessment Governing Board, 
1993); Kane, M. Comments on the NAE evaluation of the NAGB achievement levels. (Washington, DC: National Assessment 
Governing Board, 1993); NAEP Reading Revisited: An Evaluation of the 1992 Achievement Levels Descriptions. (American 
College Testing, Washington, DC: National Assessment Governing Board, 1993); Technical Report on Setting Achievement Levels 
on the 1992 National Assessment of Educational Progress in Mathematics, Reading, and Writing. (American College Testing, 
Washington, DC: National Assessment Governing Board, 1993). 
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The student achievement levels in this report have been developed carefully and 
responsibly, and the procedures used have been refined and revised as new technologies have 
become available. Upon review of the available information, the Commissioner of Education 
Statistics has judged that the achievement levels are in a developmental status. However, the 
Commissioner and the Governing Board also believe that the achievement levels are useful and 
valuable for reporting on the educational achievement of students in the United States. 
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TABLE 4 — GRADE 8 



Percentage of Students in Nonpublic Schools Attaining 
Mathematics Achievement Levels 





At or Above 


At or Above 




Advanced 


Proficient 


Basic 


Below Basic 



NonpubMc Schools 
1996 New Hampshire 


8 ( 3.3) 


43(6.5) 


86(3.2) 


14(3.2) 


Northeast 


3(2.1) 


28 ( 5.6) 


73 ( 7.0) 


27 ( 7.0) 


Nation 


6 ( 12) 


33 ( 2.9) 


75 ( 2.8) 


25 ( 2.8) 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, for each population 
of interest, the value for the entire population is within ± 2 standard errors of the estimate for the sample. In comparing two 
estimates, one must use the standard error of the difference (see Appendix A for details). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Mathematics 
Assessment. 



• The percentage of nonpublic school students in New Hampshire who performed at 
or above the Proficient level (43 percent) was not significantly different from* that 
of nonpublic school students across the nation (33 percent). 



Description of Mathematics Achievement Levels 

The three mathematics achievement levels for grade 8 are described in terms specific to 
the mathematics assessment in Figure 2. Examples of questions appropriate at each achievement 
level are also provided. It should be noted that constructed-response questions occur in the 
assessment at all levels of mathematics achievement. 



* 

Although the difference may appear large, recall that “significance” here refers to “statistical significance.” 
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FIGURE 2 






Mathematics Achievement Levels 


State Assessment 



GRADE 8 

NAEP mathematics content strands: (1) Number Sense, Properties, and Operations; 
(2) Measurement; (3) Geometry and Spatial Sense; (4) Data Analysis, Statistics, and 
Probability; (5) Algebra and Functions. 

Skills are cumulative across all levels — from Basic to Proficient to Advanced. 

Eighth-grade students performing at the Basic level should exhibit evidence 
of conceptual and procedural understanding In the five NAEP confenf 
sfrands. This level of performance signifies an understanding of arithmetic 
operations — Including estimation — on whole numbers, decimals, fractions, 
and percents. In relation to the NAEP mathematics scale, Bas/c-level 
achievement for eighth grade is defined by scale scores at or above 262. 



BASIC 

LEVEL 



Specifically, eighth graders performing at the Basic level should complete problems correctly with the help 
of structural prompts such as diagrams, charts, and graphs. They should be able to solve problems in all 
NAEP content strands through the appropriate selection and use of strategies and technological tools — 
including calculators, computers, and geometric shapes. Students at this level should also be able to use 
fundamental algebraic and informal geometric concepts in problem solving. 



As they approach the Proficient level, students at the Basic level should be able to determine which of 
available data are necessary and sufficient for correct solutions and use them in problem solving. However, 
these eighth graders may show limited skill in communicating mathematically. 



PROFICIENT 

LEVEL 



Eighth-grade students performing at the Proficient level should apply 
mathematical concepts and procedures consistently to complex problems in 
the five NAEP confenf sfrands. In relation to the NAEP mathematics scale, 
Protfc/enf-level achievement for eighth grade is defined by scale scores at or 
above 299. 



Specifically, eighth graders performing at the Proficient level should be able to conjecture, defend their 
ideas, and give supporting examples. They should understand the connections between fractions, percents, 
decimals, and other mathematical topics such as algebra and functions. Students at the Proficient level are 
expected to have a thorough understanding of basic level arithmetic operations — an understanding 
sufficient for problem solving in practical situations. 

Quantity and spatial relationships in problem solving and reasoning should be familiar to them, and they 
should be able to convey underlying reasoning skills beyond the level of arithmetic. They should be able 
to compare and contrast mathematical ideas and generate their own examples. These students should make 
inferences from data and graphs; apply properties of informal geometry; and accurately use the tools of 
technology. Students at this level should understand the process of gathering and organizing data and be 
able to calculate, evaluate, and communicate results within the domain of statistics and probability. 



ADVANCED 

LEVEL 



Eighth-grade students at the Advanced level should be able to reach beyond 
the recognition, identification, and application of mathematical rules in order 
to generalize and synthesize concepfs and principles in th * five NAEP content 
strands . In relation to the NAEP mathematics scale, Advanced- 1 eve I 
achievement for eighth grade is defined by scale scores at or above 333. 



Specifically, eighth graders performing at the Advanced level should be able to probe examples and 
counterexamples in order to shape generalizations from which they can develop models. Eighth graders 
performing at this level should use number sense and geometric awareness to consider the reasonableness 
of an answer. They are expected to use abstract thinking to create unique problem-solving techniques and 
explain the reasoning processes underlying their conclusions. 
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Mathematics Achievement Levels 


State Assessment 



Grade 8 Basic-Level Example Item 



Which of the following is both a multiple of 3 and a multiple of 7? 

A. 7,007 

B. 8,192 
*C. 21,567 

D. 22,287 

E. 40,040 



Did you use the calculator on this question? 

Yes No 



1992 Percent Correct 



Nation 



76 (1.3) 



Grade 8 Proficient-Level Example Item 





80 




70 


Number 


60 


of 


50 


Sit-ups 


40 




30 



10 15 20 25 30 
Age in Years 

In the graph above, each dot shows the number of sit-ups and the 
corresponding age for one of 13 people. According to this graph, what is the 
median number of sit-ups for these 13 people? 



A. 15 

B. 20 

C. 45 



* D. 50 


1992 Percent Correct 


E. 55 


Nation 


23 (1 .4) 



Did you use the calculator on this question? 

Yes No 
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FIGURE 2 (continued) 


Mathematics Achievement Levels 


State Assessment 



Grade 8 Advanced-Level Example Item 



A 


B 


2 


5 


4 


9 


6 


13 


8 


17 



14 ? 



If the pattern shown in the table were continued, what number would appear 
in the box at the bottom of column B next to 14? 



A. 19 

B. 21 

C. 23 

D. 25 
* E. 29 



1992 Percent Correct 



Nation 



25 (1.4) 
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APPENDIX A 



Reporting NAEP 1996 Mathematics Results 
for New Hampshire 

A.1 Participation Guidelines 

As was discussed in the Introduction, unless the overall participation rate is 
sufficiently high for a jurisdiction, there is a risk that the assessment results for that 
jurisdiction will be subject to appreciable nonresponse bias. Moreover, even if the 
overall participation rate is high, there may be significant nonresponse bias if the 
nonparticipation that does occur is heavily concentrated among certain types of schools 
or students. The following guidelines concerning school and student participation rates 
in the state assessment program were established to address four significant ways in 
which nonresponse bias could be introduced into the jurisdiction sample estimates. The 
guidelines determining a jurisdiction’s eligibility to have its results published are 
presented below. Also presented below are the conditions that will result in a 
jurisdiction’s receiving a notation in the 1996 reports. Note that in order for a 
jurisdiction’s results to be published with no notations, that jurisdiction must satisfy all 
guidelines. (A more complete discussion of the NAEP participation guidelines can be 
found in the Technical Report of the NAEP 1996 State Assessment Program in 
Mathematics.) 

Guidelines on the Publication of NAEP Results 



Guideline 1 — Publication of Public School Results 
A jurisdiction will have its public school results published in the NAEP 
1996 Mathematics Report Card (or in other reports that include all 
state-level results) if and only if its weighted participation rate for the 
initial sample of public schools is greater than or equal to 70 percent. 
Similarly, a jurisdiction will receive a separate NAEP 1996 Mathematics 
State Report if and only if its weighted participation rate for the initial 
sample of public schools is greater than or equal to 70 percent. 
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Guideline 2 — Publication of Nonpublic School Results 
A jurisdiction will have its nonpublic school results published in the 
NAEP 1996 Mathematics Report Card (or in other reports that include 
all state-level results) if and only if its weighted participation rate for 
the initial sample of nonpublic schools is greater than or equal to 70 
percent AND meets minimum sample size requirements. 1 A 
jurisdiction eligible to receive a separate NAEP 1996 Mathematics State 
Report under guideline 1 will have its nonpublic school results included 
in that report if and only if that jurisdiction’s weighted participation rate 
for the initial sample of nonpublic schools is greater than or equal to 
70 percent AND meets minimum sample size requirements. If a 
jurisdiction meets guideline 2 but fails to meet guideline 1, a separate 
NAEP 1996 Mathematics State Report will be produced containing only 
nonpublic school results. 

Guideline 3 — Publication of Combined Public and 
Nonpublic School Results 

A jurisdiction will have its combined results published in the NAEP 
1996 Mathematics Report Card (or in other reports that include all 
state-level results) if and only if both guidelines 1 and 2 are satisfied. 
Similarly, a jurisdiction eligible to receive a separate NAEP 1996 
Mathematics State Report under guideline 1 will have its combined 
results included in that report if and only if guideline 2 is also met. 

Guidelines for Notations of NAEP Results 

Guideline 4 — Notation for Overall Public School 
Participation Rate 

A jurisdiction that meets guideline 1 will receive a notation if its 
weighted participation rate for the initial sample of public schools was 
below 85 percent AND the weighted public school participation rate 
after substitution was below 90 percent. 

Guideline 5 — Notation for Overall Nonpublic School 
Participation Rate 

A jurisdiction that meets guideline 2 will receive a notation if its 
weighted participation rate for the initial sample of nonpublic schools 
was below 85 percent AND the weighted nonpublic school participation 
rate after substitution was below 90 percent. 



23 



Minimum participation size requirements for reporting nonpublic school data consist of two components: (1) a school 
sample size of six or more participating schools and (2) an assessed student sample size of at least 62. 



20 



THE NAEP 1996 STATE ASSESSMENT IN MATHEMATICS 



New Hampshire 



Guideline 6 — Notation for Strata-Specific Public 
School Participation Rate 

A jurisdiction that is not already receiving a notation under 
guideline 4 will receive a notation if the sample of public schools 
included a class of schools with similar characteristics that had a 
weighted participation rate (after substitution) of below 80 percent, and 
from which the nonparticipating schools together accounted for more 
than five percent of the jurisdiction’s total weighted sample of public 
schools. The classes of schools from each of which a jurisdiction 
needed minimum school participation levels were determined by degree 
of urbanization, minority enrollment, and median household income of 
the area in which the school is located. 

Guideline 7 — Notation for Strata-Specific Nonpublic 
School Participation Rate 

A jurisdiction that is not already receiving a notation under 
guideline 5 will receive a notation if the sample of nonpublic schools 
included a class of schools with similar characteristics that had a 
weighted participation rate (after substitution) of below 80 percent, and 
from which the nonparticipating schools together accounted for more 
than five percent of the jurisdiction’s total weighted sample of 
nonpublic schools. The classes of schools from each of which a 
jurisdiction needed minimum school participation levels were 
determined by type of nonpublic school (Catholic versus non-Catholic) 
and location (metropolitan versus nonmetropolitan). 

Guideline 8 — Notation for Overall Student Participation 
Rate in Public Schools 

A jurisdiction that meets guideline 1 will receive a notation if the 
weighted student response rate within participating public schools was 
below 85 percent. 

Guideline 9 — Notation for Overall Student Participation 
Rate in Nonpublic Schools 

A jurisdiction that meets guideline 2 will receive a notation if the 
weighted student response rate within participating nonpublic schools 
Was below 85 percent. 
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Guideline 10 — Notation for Strata-Specific Student 
Participation Rates in Public Schools 
A jurisdiction that is not already receiving a notation under 
guideline 8 will receive a notation if the sampled students within 
participating public schools included a class of students with similar 
characteristics that had a weighted student response rate of below 80 
percent, and from which the nonresponding students together accounted 
for more than five percent of the jurisdiction’s weighted assessable 
public school student sample. Student groups from which a jurisdiction 
needed minimum levels of participation were determined by the age of 
the student, whether or not the student was classified as a student with 
a disability (SD) or of limited English proficiency (LEP), and the type 
of assessment session (monitored or unmonitored), as well as school 
level of urbanization, minority enrollment, and median household 
income of the area in which the school is located. 

Guideline 11 — Notation for Strata-Specific Student 
Participation Rates in Nonpublic Schools 
A jurisdiction that is not already receiving a notation under 
guideline 9 will receive a notation if the sampled students within 
participating nonpublic schools included a class of students with similar 
characteristics that had a weighted student response rate of below 80 
percent, and from which the nonresponding students together accounted 
for more than five percent of the jurisdiction’s weighted assessable 
nonpublic school student sample. Student groups from which a 
jurisdiction needed minimum levels of participation were determined 
by the age of the student, whether or not the student was classified as 
a student with a disability (SD) or of limited English proficiency (LEP), 
and the type of assessment session (monitored or unmonitored), as well 
as type and location of school. 
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A.2 NAEP Reporting Groups 

The state assessment program provides results for groups of students defined by 
shared characteristics — region of the country, gender, race/ethnicity, parental education, 
location of the school, type of school, participation in Title I programs, and eligibility 
for the free/reduced-price lunch component of the National School Lunch Program. 
Based on criteria described later in this appendix, results are reported for subpopulations 
only when sufficient numbers of students and adequate school representation are present. 
For public school students, the minimum requirement is at least 62 students in a 
particular subgroup from at least 5 primary sampling units (PSUs). 2 For nonpublic 
school students, the minimum requirement is 62 students from at least 6 different schools 
for the state assessment program or from at least 5 PSUs for the national assessment. 
However, the data for all students, regardless of whether their subgroup was reported 
separately, were included in computing overall results. Definitions of the subpopulations 
referred to in this report are presented on the following pages. 

Region 

Results are reported for four regions of the nation: Northeast, Southeast, Central, 
and West. States included in each region are shown in Figure A.l. All 50 states and 
the District of Columbia are listed. Territories and the two Department of Defense 
Educational Activities jurisdictions were not assigned to any region. 

Regional results are based on national assessment samples, not on aggregated state 
assessment program samples. Thus, the regional results are based on a sample that is 
different and separate from that used to report the state results. 
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FIGURE A.l 




M 


Regions of the Country 


State Assessment 



NORTHEAST 


SOUTHEAST 


CENTRAL 


WEST 



Connecticut 


Alabama 


Illinois 


Alaska 


Delaware 


Arkansas 


Indiana 


Arizona 


District of Columbia 


Florida 


Iowa 


California 


Maine 


Georgia 


Kansas 


Colorado 


Maryland 


Kentucky 


Michigan 


Hawaii 


Massachusetts 


Louisiana 


Minnesota 


Idaho 


New Hampshire 


Mississippi 


Missouri 


Montana 


New Jersey 


North Carolina 


Nebraska 


Nevada 


New York 


South Carolina 


North Dakota 


New Mexico 


Pennsylvania 


Tennessee 


Ohio 


Oklahoma 


Rhode Island 


Virginia* 


South Dakota 


Oregon 


Vermont 

Virginia* 


West Virginia 


Wisconsin 


Texas 

Utah 

Washington 

Wyoming 



* The part of Virginia that is included in the Washington, DC, metropolitan area is included in the Northeast region; the 
remainder of the state is in the Southeast region. 




2 



For the State Assessment Program, a PSU is most often a single school; for the national assessment, a PSU is a selected 
geographic region (a county, group of counties, or a metropolitan statistical area). 
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Gender 

Results are reported separately for males and females. 

Race/Ethnicitv 

The race/ethnicity variable is derived from two questions asked of students and 
schools’ records, and it is used for race/ethnicity subgroup comparisons. Two questions 
from the set of general student background questions were used to determine 
race/ethnicity: 



If you are Hispanic, what is your Hispanic background? 
° lam not Hispanic. 

° Mexican, Mexican American, or Chicano 
° Puerto Rican 
° Cuban 

° Other Spanish or Hispanic background 



Students who responded to this question by filling in the second, third, fourth, or 
fifth oval were considered Hispanic. For students who filled in the first oval, did not 
respond to the question, or provided information that was illegible or could not be 
classified, responses to the question below were examined in an effort to determine 
race/ethnicity. 



Which best describes you? 

° White (not Hispanic) 

° Black (not Hispanic) 

° Hispanic (“Hispanic” means someone who is from a Mexican, 
Mexican American, Chicano, Puerto Rican, Cuban, 
or other Spanish or Hispanic background.) 

° Asian or Pacific Islander (“Asian or Pacific Islander” 
means someone who is from a Chinese, Japanese, Korean, 

Filipino, Vietnamese, or other Asian or Pacific Island background.) 

° American Indian or Alaskan Native (“American Indian or 

Alaskan Native” means someone who is from one of the American 
Indian tribes, or one of the original people of Alaska.) 

° Other (specify) 
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Students’ race/ethnicity was then assigned on the basis of their response. For 
students who filled in the sixth oval (“Other”) or provided illegible information or 
information that could not be classified, or did not respond at all, race/ethnicity was 
assigned as determined by school records . 3 

Race/ethnicity could not be determined for students who did not respond to either 
of the demographic questions and whose schools did not provide information about 
race/ethnicity. 

The details of how race/ethnicity classifications were derived is presented so that 
readers can determine how useful the results are for their particular purposes. Also, 
some students indicated that they were from a Hispanic background (e.g., Puerto Rican 
or Cuban) and that a racial/ethnic category other than Hispanic best described them. 
These students were classified as Hispanic based on the rules described above. 
Furthermore, information from the schools did not always correspond to how students 
described themselves. Therefore, the racial/ethnic results presented in this report attempt 
to provide a clear picture based on several sources of information. 

Parents* Highest Level of Education 

The variable representing level of parental education is derived from responses to 
two questions from the set of general student background questions. Students were 
asked to indicate the extent of their mother’s education: 



How far in school did your mother go? 

° She did not finish high school. 

° She graduated from high school. 

° She had some education after high school. 
° She graduated from college. 

° I don’t know. 



Students were asked a similar question about their father’s education level: 

How far in school did your father go? 

° He did not finish high school. 

° He graduated from high school. 

° He had some education after high school. 

° He graduated from college. 

° I don’t know. 



3 The procedure for assigning race/ethnicity was modified for Hawaii. See W Qjchnical Report for the NAEP 1996 State 
Assessment Program in Mathematics for details. 
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The information was combined into one parental education reporting variable 
determined through the following process. If a student indicated the extent of education 
for only one parent, that level was included in the data. If a student indicated the extent 
of education for both parents, the higher of the two levels was included in the data. If 
a student did not know the level of education for both parents or did not know the level 
for one parent and did not respond for the other, the parental education level was 
classified as “I don’t know.” If the student did not respond for either parent, the student 
was recorded as having provided no response. (Nationally, 36 percent of fourth graders 
and 1 1 percent of eighth graders reported that they did not know the education level of 
either of their parents.) 

Type of Location 

Results are provided for students attending public schools in three mutually 
exclusive location types — central city, urban fringe/large town, and rural/small town 
— as defined below. The type of location variable is defined in such a way as to indicate 
the geographical location of a student’s school. The intention is not to indicate, or 
imply, social or economic meanings for these location types. The type of location 
variable, on which the current NAEP sampling is based, does not support the reporting 
of regional results. Therefore, only state and national results will be presented. 

Central City. The Central City category includes central cities of all 
Metropolitan Statistical Areas (MSAs). 4 Central City is a geographic 
term and is not synonymous with “inner city.” 

Urban Fringe/Large Town: An Urban Fringe includes all densely 

settled places and areas within MSAs that are classified as urban by the 
Bureau of the Census. A Large Town is defined as places outside 
MSAs with a population greater than or equal to 25,000. 

Rural/Small Town: Rural includes all places and areas with a 

population of less than 2,500 that are classified as rural by the Bureau 
of the Census. A Small Town is defined as places outside MSAs with 
a population of less than 25,000 but greater than or equal to 2,500. 
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Each Metropolitan 



Statistical Area 



(MSA) is defined by the Office of Management and Budget. 
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Type of School 

Samples for the 1996 state assessment program were expanded to include students 
attending nonpublic schools (Catholic schools and other religious and private schools) 
in addition to students attending public schools. The expanded coverage was instituted 
for the first time in 1994. Samples for the 1990 and 1992 Trial State Assessment 
programs had been restricted to public school students only. For those jurisdictions 
meeting pre-established participation rate standards (see earlier section of this appendix), 
separate results are reported for public schools, for nonpublic schools, and for the 
combined public and nonpublic school samples. The combined sample for each 
jurisdiction also contains students attending Bureau of Indian Affairs (BIA) schools and 
Department of Defense Domestic Dependent Elementary and Secondary Schools 
(DDESS) in that jurisdiction. These two categories of schools are not included in either 
the public or nonpublic school samples. 

Note that the DDESS and Department of Defense Dependents Schools 
(DoDDS) 5 were assessed in 1996 as separate jurisdictions, reported as jurisdictions with 
public school samples only. 

Title I Participation 

Based on available school records, students were classified as either currently 
participating in a Title I program or receiving Title I services, or as not receiving such 
services. The classification applies only to the school year when the assessment was 
administered (i.e., the 1995-96 school year) and is not based on participation in previous 
years. If the school did not offer any Title I programs or services, all students in that 
school were classified as not participating. 

Eligibility for the Free/Reduced-Price School Lunch Program 

Based on available school records, students were classified as either currently 
eligible for the free/reduced-price lunch component of the Department of Agriculture’s 
National School Lunch Program or not eligible. The classification refers only to the 
school year when the assessment was administered (i.e., the 1995-96 school year) and 
is not based on eligibility in previous years. If school records were not available, the 
student was classified as “Information not available.” If the school did not participate 
in the program, all students in that school were classified as “Information not available.” 

A.3 Guidelines for Analysis and Reporting 

This report describes mathematics performance for eighth graders and compares 
the results for various groups of students within these populations — for example, those 
who have certain demographic characteristics or who responded to a specific background 
question in a particular way. The report examines the results for individual demographic 
groups and individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 



The Department of Defense Dependents Schools (DoDDS) refers to overseas schools (i.e., schools outside the United 
States). Department of Defense Domestic Dependent Elementary and Secondary Schools (DDESS) refers to domestic 
schools (i.e., schools in the United States). 
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Drawing Inferences from the Results 

Because the percentages of students in these subpopulations and their average 
scale scores are based on samples — rather than on the entire population of eighth 
graders in a jurisdiction — the numbers reported are necessarily estimates. As such, they 
are subject to a measure of uncertainty, reflected in the standard error of the estimate. 
When the percentages or average scale scores of certain groups are compared, it is 
essential to take the standard error into account, rather than to rely solely on observed 
similarities or differences. Therefore, the comparisons discussed in this report are based 
on statistical tests that consider both the magnitude of the difference between the 
averages or percentages and the standard errors of those statistics. 

One of the goals of the state assessment program is to estimate scale score 
distributions and percentages of students in the categories described in A. 2 for the 
overall populations of fourth- and eighth-grade students in each participating jurisdiction 
based on the particular samples of students assessed. The use of confidence intervals, 
based on the standard errors, provides a way to make inferences about the population 
average scale scores and percentages in a manner that reflects the uncertainty associated 
with the sample estimates. An estimated sample average scale score ± 2 standard errors 
approximates a 95 percent confidence interval for the corresponding population average 
or percentage. This means that one can conclude with approximately 95 percent 
confidence that the average scale score of the entire population of interest (e.g., all 
fourth-grade students in public schools in a jurisdiction) is within ± 2 standard errors 
of the sample average. 

As an example, suppose that the average mathematics scale score of the students 
in a particular jurisdiction’s eighth-grade sample were 256 with a standard error of 1.2. 
A 95 percent confidence interval for the population average would be as follows: 

Mean ± 2 standard errors = 256 ± 2 x (1 .2) = 256 ± 2.4 = 

256 - 2.4 and 256 + 2.4 = (253.6, 258.4) 

Thus, one can conclude with 95 percent confidence that the average scale score 
for the entire population of eighth-grade students in public schools in that jurisdiction 
is between 253.6 and 258.4. 

Similar confidence intervals can be constructed for percentages, if the percentages 
are not extremely large or extremely small. For extreme percentages, confidence 
intervals constructed in the above manner may not be appropriate, and accurate 
confidence intervals can be constructed only by using procedures that are quite 
complicated. 

Extreme percentages, defined by both the magnitude of the percentage and the size 
of the sample from which it was derived, should be interpreted with caution. (The 
forthcoming Technical Report of the NAEP 1996 State Assessment Program in 
Mathematics contains a more complete discussion of extreme percentages.) 




28 



THE NAEP 1996 STATE ASSESSMENT IN MATHEMATICS 



New Hampshire 



Analyzing Subgroup Differences in Averages and Percentages 

The statistical tests determine whether the evidence — based on the data from the 
groups in the sample — is strong enough to conclude that the averages or percentages 
are really different for those groups in the population. If the evidence is strong (i.e., the 
difference is statistically significant), the report describes the group averages or 
percentages as being different (e.g., one group performed higher than or lower than 
another group) — regardless of whether the sample averages or sample percentages 
appear to be about the same or not. If the evidence is not sufficiently strong (i.e., the 
difference is not statistically significant), the averages or percentages are described as 
being not significantly different — again, regardless of whether the sample averages or 
sample percentages appear to be about the same or widely discrepant. The reader is 
cautioned to rely on the results of the statistical tests — rather than on the apparent 
magnitude of the difference between sample averages or percentages — to determine 
whether those sample differences are likely to represent actual differences between the 
groups in the population. 

In addition to the overall results, this report presents outcomes separately for a 
variety of important subgroups. Many of these subgroups are defined by shared 
characteristics of students, such as their gender or race/ethnicity and the type of location 
in which their school is situated. Other subgroups are defined by the responses of the 
assessed students’ mathematics teachers to questions in the mathematics teacher 
questionnaire. 

In Chapter 1 of this report, differences between the jurisdiction and the nation were 
tested for overall mathematics scale score and for each of the mathematics content areas. 
In Chapter 2, significance tests were conducted for the overall scale score for each of 
the subpopulations. Chapter 3 reports differences between the jurisdiction and nation 
for the percentage of students at or above the Proficient level, and Chapter 4 contains 
significance tests for the percentage of students at or above the Proficient level for each 
of the subpopulations. In Chapters 5 through 7, comparisons were made across 
subgroups for responses to various background questions. 

As an example of comparisons across subgroups, consider the question: Do 
students who reported discussing studies at home almost every day exhibit higher average 
mathematics scale scores than students who report never or hardly ever doing so? 

To answer the question posed above, begin by comparing the average mathematics 
scale score for the two groups being analyzed. If the average for the group that reported 
discussing their studies at home almost every day is higher, it may be tempting to 
conclude that that group does have a higher mathematics scale score than the group that 
reported never or hardly ever discussing their studies at home. However, even though 
the averages differ, there may be no real difference in performance between the two 
groups in the population because of the uncertainty associated with the estimated average 
scale scores of the groups in the sample. Remember that the intent is to make a 
statement about the entire population, not about the particular sample that was assessed. 
The data from the sample are used to make inferences about the population as a whole. 
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As discussed in the previous section, each estimated sample average scale score 
(or percentage) has a degree of uncertainty associated with it. It is therefore possible 
that if all students in the population (rather than a sample of students) had been assessed 
or if the assessment had been repeated with a different sample of students or a different, 
but equivalent, set of questions, the performances of various groups would have been 
different. Thus, to determine whether there is a real difference between the average 
scale score (or percentage of a certain attribute) for two groups in the population, an 
estimate of the degree of uncertainty associated with the difference between the scale 
score averages or percentages of those groups must be obtained for the sample. This 
estimate of the degree of uncertainty — called the standard error of the difference 
between the groups — is obtained by taking the square of each group’s standard error, 
summing these squared standard errors, and then taking the square root of this sum. 

In a manner similar to that in which the standard error for an individual group 
average or percentage is used, the standard error of the difference can be used to help 
determine whether differences between groups in the population are real. The difference 
between the mean scale score or percentage of the two groups — 2 standard errors of 
the difference — represents an approximate 95 percent confidence interval. If the 
resulting interval includes zero, there is insufficient evidence to claim a real difference 
between groups in the population. If the interval does not contain zero, the difference 
between groups is statistically significant (different) at the .05 level. 

As another example, to determine whether the average mathematics scale score 
of fourth-grade males is higher than that of fourth-grade females in a particular 
jurisdiction’s public schools, suppose that the sample estimates of the average scale 
scores and standard errors for males and females were as follows: 



Group 


Average Scale Score 


Standard Error 


Males 


218 


0.9 


Females 


216 


1.1 



The difference between the estimates of the average sale scores of males and 
females is two points (218 - 216). The standard error of this difference is 

V 0.9 2 + EF = 1.4 

Thus, an approximate 95 percent confidence interval for this difference is 
Mean difference ± 2 standard errors of the difference = 



2 ± 2 x (1.4) = 2 ± 2.8 = 2 - 2.8 and 2 + 2.8 = (-0.8, 4.8) 
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